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# W j; #'J tit W » 

( «t m * ^ » # 3. #a & ^ • It £f 4* t- £ & • * & ffe $P g ) 

^ f tfrS#J : ^IPC • 

tt*p«rH6*^W 3S«8-ft ft * « a A ^. ^ JR 4/ 

METHOD AND APPARATUS FOR REMOVING BLOCKING ARTIFACT OF VIDEO PICTURE VIA 

* 

LOOP FILTERING USING PERCEPTUAL THRESHOLD 

# A.'(*U) ' 

^iH-ttASt^PR-^g/ALI CORPORATION 
>ft*A : (tX/^i) (#30 SSit/ LU, TEDDY 

^Jbffr^^^- J3t— raj^Sfe— 2F, No. 246, Sec. 1, Nei-Hu Rd. , Taipei City 
114, Taiwan, R. 0. C. 

m # : ( tX/^X) *#R,H/TW 

- *9iA : (*3 A) 

1. 4fgj&/ KIM, CHANGSUNG 

2. SHIH, HSUAN-HUEI 

3. KUO, CHUNG-CHI EH 

1. ^JbTp-rt^-S:— ra^Sfe— ^/ 2F, No. 246, Sec. 1, Nei-Hu Rd. , Taipei 
City 114, Taiwan, R.O.C. 

2. fHtTFftMl|*£--ft— ra^Sfc— 2F, No. 246, Sec. 1, Nei-Hu Rd. , Taipei 
City, Taiwan, R.O.C. 
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□ Si*m±4*7 [*8-s£*jH& : 3P#*Mfc : a #J '• ft<* motile.] 
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^J^S — l^lfea®^MlE^&2.K^lS%5SJiS (blocking artifact) - 
(boundary edge pixel) WfcfcW^W&^M i&^f-W^ £#frt 



The present invention provides an image processing method and 
related device for processing blocking artifacts between two blocks 
of a video picture with loop filtering. The present invention 
compares two boundary edge pixels at both sides of a boundary between 
two blocks according to at least one perceptual threshold to 
determine if pixel values of the two boundary edge pixels should be 
adjusted to decrease the difference corresponding to the pixel values 
of the two boundary edge pixels. 
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mmrnm ' M^frmiMmW\2.mmmM ( P Frame > Predictive Frame ) 

^mmmmmm no wmmm& i3omMm^mmmmm^mmm 

m ( I Frame * Intra Frame ) mfrttM ' &%f&W§m®&M2Jffi&W& 

m ° mmmmfc^ n6 * 136 wm^mmm^ uommmm^ 130 n 



is • agm&gi&€P/££ h.264 iswmp^is • \>x— h.264 

ZMmtm ( blocking artifact ) ° : 

MaftJ'Mi^ ( boundary edge pixel ) ^W^M^MMMMjA^-^M^^. 




n 



2MkftWfflj£ffl&!jf&M ( boundary edge pixel ) ^fcfeM&M&MW 

mm^m%mz.mmmmmmz.mm. ; j^— mw^c c arithmetic 
unit) . mmm^mtm^7ummmww-7t » ji^f«E£m»imite 

201a * 201b £P»LfM • *«^5l^"ir5fe5^a[ 200 ^TOggfiB 
( frame type ) > grff 200 #^H#m^P • ai5#flKS«jTOg#^ 
— ftlMP.fl (Intra frame) Bf • #lfr#H 201a ■ ^M(J > ^gftuff^S 
Iftl^g^— ilKllHfll (Inter frame) > gfe#tfj^ 201b • _h2tt^|?3 
IMimMMIJIW : ^Kiitl^ (Intra slice) MfflfrftMMJn&i (SI 
slice > Synchronized Intra slice ) • M ±^^MM^W^M^JWMMM : ?1 
SdilSflf Jt*I8 (P slice > Predicted slice) » IrJ WMWMfn&t (B slice - 
Bidirectional predicted slice ) > ^W\#WMWMP\Wt ( SP slice > 
Synchronized Predicted slice ) - £S^f£ 201b iftp^&i^-fjl 201a 

^fflicfffiitt • zATmvmm^m. 201a . ^g^^s 201b - 



12 









fflfa ( macroblock ) 300 MffiS&2.WfiL ' UWMM 300 M W+Tn 






348 ' 3t*& 




jif 3|8J.( chromatic 



(block) 315 - 316 

luminance pixel data )l2x2 
pixel data) - BA^ttftgfeBWfe**^^^ ' MB/n^I fftlljft 








P ^ Q (*«7R^BBffl7F) £i*HU0l . 



= > mm * mm/Km^i^mmn p { > * a = o ^ 1 ) &m 

q *wmmm\*im n ±mmmmmmm p > q 401 j^awfc* 

( boundary edge pixel ) $ftm ' Mi^S*4 Pi ' qi WlfrWSft£&&m 
IlfRli n ±M^M15M^- 401 2_-$8§LR$m ( interior edge pixel ) #J 

mm ' «b« • ^Mis^a 326 * 336 p * 




Am^si?- 401 wnm^pn^zm^ 303 . M&iii/Nm^m 

Spjf^^SHlRl* m « itt0tffl/N^#^»4 po > pi 

326 4>3Bffift#jl 303 303 
IrJS m _L;£#?jtf^^4 ' MW/\2&MWft qo ' qi 
f^gi£^ 336 tfJKfiSftJF- 303 m®f^TO«ai^ 303 ^[pj 

315 - 316 > 348 




mwmm 




:irj 



m 






E^BE% 300 6*J^- 




301 - 302 - 




308 ±tfjl&MMm ( blocking artifact ) rJS 





$P HI —^TTF ' ^ 

^MUE^t P * Q 




/ill 
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mmmmmmmm* 



202 
204 



^JHE 206 



^M. 208 



£BB 210 




-be 201a zm mmtuT • 






<3s 

mwi-T^im^m\^m P ' Q Pi * q; ( i = 0 > 1 » 2 ) ; 

wHUWMMF^HK Noticeable Difference Threshold ) AI 

f£&KMlB$l p^Q ^ift^ 401 Wfc^Jl^tefl 

*4 po • q 0 ^.^m^mmmmm^mm^mmmm p 0 * q 0 
^mmmmMtmz-mmmn p 0 * q 0 mmm^mm i Po - q 0 
/Jnj$wfs wngfitAi > mmxmm&± c Eari y . 

Termination ) %m 290 J^fP^Bf f^^S^^^ jPfBf. 

^mmx&m 206 » 





( JND * Just Noticeable Difference ) AI ; 



WM—nSffi^^ffi'^gM ( Recognizable Discontinuity 
Threshold ) T(AQ, M)kh$£MWWiM P * Q 401 

n-^m^Mmmn p 0 - q 0 jw&ms 9 * 
mmz.mmmm p 0 > q 0 • ^mmmm^m^i 
qommm^mM i p 0 -q 0 1 /j^RT^ii^rwe 





HI4 Po x 



T(AQ, AI) » SOU A#BS 208 ' 5IU*IAffi¥*£lfciifl!8 290 





T(AQ, AI)^Jn BT|g^Rjp^^HRK0J ( Recognizable 
Discontinuity Limit ) T(AQ, AI) ; 

tBSI— WS&H^M ( Adjustment Threshold ) Aq/21±M^M 

if«*4»^:-Ha«^:«t3Mm po is q 0 m-mm^mm 
m^mmmm- pi ^ q, ^^m^mmmmmm^mjti 

W& Po * qo ' P ZMM'MPc^Si I Pi- Po I /J^ii 

o/2^M^ Q I qi- qo I /.hSSfBOT 






mmo/2 » jma#§h 210 . siusiAji^jfcjwB 290 



zmmmm p 0 * qo«^it 1 Po - q 0 





I Po " 
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qo I f^l^lliilltfl ( luminance difference ) ^"fe^ 
( chromatic difference ) ° ^^§^fMli£ln:^$D~F : 




Po' = po + kA 0 



q 0 ' = q 0 - kA 0 



t^Wk 212p 




&#5!l$tfT#ER 212p * 212q ; 








mm pi ^^m.mm.m^m.m-m.m^.mm'Mm pi » 
* I pi-po' I A^kmmmmnmuM - «a^wiH£ 



2i4p ; 

&m 2i4 P : it nmmm^mnmm^i 




HZ. 





Pi 



= Pl + o.5kA 0 > mRttMmya&^mzLii 

pi 'M#^^iif 








fift* pi IMS 



m^Mm^mz.mmmm pi » 




M I Pl'-p 



wmx&m 2i6 P 



^S'JttSRim Pi tff 

*TO Po' #14 





^» 2i6 P : mm^mm^mzi 

= pi + 0.5kA 0 i 




f4pieJ 







Pi 



#fH 212q : tMSW^^P^flAI it«KW*#^*;2: 






3F4 qo'|S|-«^^iftM»^ 



HI4 qi ^^(l^mS^tlS^im q 





:i ' ^6" 

» I qi-qo' i 'hnmmmmrmmM • i!MA^#M&ifj£ 

It 291q i^fp^Bf ra25f5BKa^W5H*3Rim > ^J«A#J 
214q ; 

#85 214q : tf-^^15^^^#^^f^^^ 






!fil q, ' 

= qi - o.5kA 0 • ^L^t\Mm^Mm^mz.mmmm qi 
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±mi.mmmmnmmi m&mm&izM c weber's Law ) m^m • 

m^WiMMMi ■ if*P^fI ( increment threshold ) fW^M^tfc 

i P * i q • f»mffi^^ ' ^e^^j^s i P m ' aissi^ Rr^?^Bs# 

PMffiAI fSSMMt P ' Q ^E^lW^AIp > AI q ^zpi%fS(AI p + 
AI q )/2 o -fegfejliaji^M^P = (AI P + AI q )/2 = (kl p + kI q )/2 - tlUWi 

so^^jt i P i q ' w&Mw'MfemfwmM^^n^s.WM = (ai p + Ai q )/2 

= (kl p + kI q )/2 - 

$U« 216p ffim ' ^^5t^f»^f^^4 Pl 0.5kA 0 ^ 

210 m^mm^mzmmmn p 0 ^mgji kAo5:^-xjp#« 
2i6q mat ' m^m.mntmzMm^MPc qi ^nn^*-o.5kA 0 s^ 210 m 

fflH^#M^^^7TK : RT^^P^il T(AQ, AI)#^^S^MM^ 
P * Q £.M^-<k0Wt ( Quantization Parameter) £MlAQ UtfCTMW! 
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W ' RlW^»MPg«EfiS T(AQ, AI)-fenJJ^^AQ §5|AI ^MM^^M^ 

t(aq, Ai) ^MWp>M5.#^MlS^P^Q^a^4bf«^MflAQ 

isk^w^m^ai wj?m » Rn#^«p«fi t(aq, ai) 

flAQ ^c^M^SAI if TO . if jjp Pl^illf P^fS T(AQ, AI) ; 

&;&^M1E8& p * Q £*^fc«teM<!AQ ^^m^^Ai W/P 

m ' M^«j^^»iP«fi t(aq, ai) . 



(BS • Boundary Strength) ffli^t . ^nfn^^^BS^S^m 

a&ai > jvt (H.264)mmmfem • iinm—ffiTv; » 

mmmum^m 201b ^nfflfjsia^T ° 

#,^202 : iflliMP'Qaiimpi * qi 0 = 0 * 1 * 2) ; 

i&m 206' : si-pM t ttmmmmm p - q 401 mf jj^m 

p 0 * q 0 - ^mm^&m^m^m^n p 0 * q 0 m 

^208' : mm-$Mmnmm^o/2ttmmmmm^m^~mm^.m 
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^fej^mnmmmmmmk^mm^ p 0 * q 0 » p 
^nfe^s^M* i pi- po i 'hnmM.nm.m*o/2WMn q 2. 
titi4Mi i qi- qo i 'hmwmnm.m*o/2 > mohair 
210' » ^lOiiA^^ihwj 290 i^ip^0tr^^«m« 






I jul 



0 




si* 




#H210' : ilill^f i^f if!4 Po - qoi^M^ 

11^4 Po > qo ^/T^f 15.11 I po - qo I ' 




mm 




2& 






Po- 



qo 





Po' = Po + kA 0 
q 0 ' = q 0 - kA 0 





m ' IWi^3»mM#t p * Q 401 » ^m^n^M 



^6oo^7k«hi . n+nn-i — ^sun— mm m ^l5^hki»«»[» 
700 ^— mm mmmm soo . ^m^^to - fi^*«^ifl§7HA 

g^^WP^^MP^AI |S|nJ^^»ir^fl T(AQ, AI) > H 
Jttimftt&$ 600 ( PLF > Perceptual Loop 

Filter) 600 - jt§tt»&gg 600 : — ffi?*7C 610 > /B3fcfS^£fllg 

mmum, p * q ^MmMPc ^ * q ; : j^s— thKH7c 620 > mmm^m 

Wtc 610 » J3^f»— jP¥££rWfiiAi I«lMM£t P - Q 
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401 mmzmmmim2_mmmm po - q 0 ■ mm^wmmm^mn. 
m^mmmn p 0 * q 0 • ^mmm^mmz-mmmm po * q 0 mmm^.mm. 

I Po-qo I />Jf^raBgW9KfilAI ' SOltKJ^TC 620 204 tfftfc ' 

&%M*M^1fe±#m 290 J^fp^Bf ' SI'J 

206 fiftl ' tt8»7C 620 RT^StlWPMM T(AQ, AI)tb 

mmm^iM p * q 401 mfemaimst^fi^tM- p 0 * q 0 

$r Po ' qoffiSHiI£ffll I Po-qo I /J\^#^*I»«fiI T(AQ, AI) > 10 

ttt«7E 620 #^$n#§£ 208 mnm—^tbm • si!iit*»7G 620 ^ 

mas . tfdnim p 0 * q 0 nstmdtmz.mmmm p 0 - q 0 > &nj&m&s&:£ 
mmmm p 0 - q 0 • Hnt_t*£M* 1 po-qo 1 TOiim^i 

S o 




600 ^fe^W • 




7C ( arithmetic unit ) 630 » U 








7C 620 #a^#H7t; 610 > Jf ^§MM^m»imf[tel 



n Pi * qi • $p^sn 208 mm ' ttRHTc 620 ^mm— wm^mu^att 



z.mmm% pi m qi ^»ystsiw^»#^^*ii^ po * q 0 
p Ttmrnmnmrn i p t - p 0 1 'hnmMnmmAo/2WM$i q j£* 





mmm. i q,-q 0 1 ^hnmmnmmAon^wittmmTteio^mum 






210 mat . i^iffTc 630 mggffig&IM 

mmm.mnm^Mmmm po - q 0 w^i i p 0 -q 0 



14 Po ' qo PJMfr 





7C620 



^njmi^Wt 212p > 214p > i^^^D^l? 216p flf^ 





mmzmmmm pi 




jc 630 if ^IS^iMljiMI pi' = pi + 

o.5kA 0 • $ps(f®msfc ' m^mm^m^mmm pi o.5kA 0 ^m 
210 mm^mmmkZMmmn po ^im kA 0 ^^ • mm& • it 





7U 620 rJWt^R 212q * 214q » J^^tSft]^ 216q ^affi 

% m-Amm^mz.mm 7 Mm qi ^uuwtg 630 m^ssm 





qi 
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= qi - o.5kA 0 • mummfm ■ m^mm^mm 




14 qi ^MWkJk 



-o.5kA 0 m^ 210 mateawife*^ 




^5 



^4 qo ^LsMMt-kAo^^ 




t(aq, ai) ; p * q zM^m&z&mAQ mmm 

ffiAi ffi&rn ' tb^TG 620 «^pnff^aJMPg«fig T(AQ, AI) - 




JDL 



tinw^jj&zmm ■ *»^*^#^s^fr 206' * 208' * 210' 











jut 0 *3 



o ^ 

0 5^ 







T 





^ h.264 mmmmmmwmmmMmmmtmBm&m - 



lap 




■0 




7E ' 
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L HJ 5Va M 









7\ H _ . ^ft^fy^O ^itk if: 




1=1 aC7Tn^ 




110, 130, 700, 800 fMt 




in, 113, 131, 133 mx/ms$mT 



112 Wi^W-Tt 



114 






7C 



116, 136, 600 iH 





120 



* 
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« 



1 10 faJLlKBL 


Til — if— 


300 




301-308,401 Iftl?. 


315-348 




610 M# 


f% 


620 tbR^TC 


630 MM 


* 


m, n 




Pi , qi ft*^ 
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i. 



■W&k 




( blocking artifact ) > Mi^fl^STj 




*4 : JUS 





— P«ffith^M^5l^M{|IJ^m5t#f^ ( boundary 



edge pixel ) m^^mmmmmm^m^mmn » 






' ill Hi^MiftJ?-f= 




5m 








( luminance difference ) l^fe^li: Jl| ( chromatic difference ) ° 
^I^Mjl^-f^^ 1 ^— f^l^l— (interior edge pixel) 

4. $n^lf TOf efflm 3 TOat^^ffi ' ^H^W : ^g— S5=P»ffiit 



fg o 

#^MM£t;£*^-bf£lfc ( quantization parameter ) ^MfS^f ' if 
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H&JMtffi^ (Weber's Law) ffi£*£HS#rWt^ (just 
noticeable difference ) ' if 1}\M.W>— HffiM » J^R 



( Weber's Law ) ff[^Mj^_Wa^-WM^^ (just noticeable difference ) ° 



io. mmmmmmm i mmmt^m • ^ 







/ KM 




'/II 



ii. —mm^mM^m^M 



viea 7J 



P.CJ 




TOtSM ( blocking artifact ) > g£iIg&»^3§^W 





IS ijt^oi^Mf O^Mitl'Mi^t ( boundary edge pixel ) ^^^g^fi 









■/til-*- 






M-W-JC ( arithmetic unit ) ' U 




( luminance difference ) ( chromatic difference ) 



13. $n^sf mm mm n ^3tb£3@lftf&$ - £*mbl»7GB 
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( interior edge pixel ) ^fl^IMMl#ti^| 




m 



i4. tiummmmm 13 wtrnzmsfimm » ^^^biOTT^i* 








15. $o^it pug 14 iimat^^^ ' ^^^mnp^fg^^-pg 




i6. ^n^iffWJiEH^ umffi&zmmti&m » n^w^mm-mm^. 




17. $n^it wjfgsim 11 » ^aiwis^*^ 

(quantization parameter) ^.^fiiiJrJHf ' ^tb^JCWnllH— P! 





jm. 



is. wmmmmmm n to^me^*?^ » M*n-mm<&%. 

(Weber's Law) ffifemZ.W a 





(just noticeable difference ) t\ 

mm » mtrnw-Ttmumm— nmm : tsaifm^ > eat 

TcM^m-P^ffi ° 





19. WffWJfOTfil 11 W/r^ME&Mi 






I JUL 



timm&jm- (weber'sLaw) jffi££j£Hs#w* 

difference ) ° 




(just noticeable 



20. mmmmmmm n - ^m&% 
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